CRIM Notebook Paper - TRECVID 2008 Video Copy Detection Using Latent Aspect Modeling Over SIFT Matches

Approach we have tested in our submitted runs. Our approach consists in finding links between video shot key-frames, based on the use of a probabilistic latent space model over local matches between the keyframe images. This allows the extraction of significant groups of local matching descriptors that may represent common characteristic elements of near duplicate key-frames. This is combined with various preprocessing steps designed to accelerate and improve the matching process for any query type, as well as post-processing steps designed to accurately find the copied video segment borders. We have submitted 3 runs. The first run (Run1) uses an automatic cropping process, a local descriptor filter and a global characteristic filter as part of the pre-processing phase. A RANSAC-based post-processing step is applied on the time code of the detected key-frames copy. A video insert detector was added for the second run (Run2). The third run (Run2Faster) is the same as the second run with the use of smaller images. Differences we found among the runs: No significant differences were found amongst the three different runs. Unfortunately, a file manipulation error resulted in the second and the third run being deprived of receiving the correct video insert filter detection output. Therefore, these runs received no video insert detection response as input, causing some video insert copy detections undetectable. Relative contribution of each component of our approach: The probabilistic latent space model over local matches between the key-frame images produces a fast, robust and accurate filtering process in relation to all possible local matches. This approach works well even if there are only a few local matches between the key-frames of the copied video in question. Therefore, only a limited number of local descriptors are necessary, resulting in a more robust copy detection process. Unfortunately, the large number of local matches still makes the process rather time consuming. What we learned about runs/approaches and the research question(s) that motivated them: Approaches based on local descriptor matching are efficient for the copy detection task. It is robust to many transformations. However, these approaches are not efficient for some query categories. For instance, the flipped query type provides totally different local descriptors and the insert query requires the use of local descriptors at a higher-resolution level. Also, local descriptor matching is very time consuming. It would then be interesting to combine local descriptor matching approaches with global characteristic comparison approaches to provide a prior knowledge about the query type. Global characteristic comparison approaches alone are not efficient enough for accurate copy detection.